Click any tag below to further narrow down your results
Links
This page describes the Artificial Analysis Decision Index, a weighted benchmark that scores small "decision models" on fast classification tasks like routing support tickets and rating urgency. It combines six evaluations, from decision-making and tool calling to hallucination and answer invariance, into one composite score.
- Decision models answer every question in a single call with a probability for each option, which keeps them cheap and fast and makes calibration measurable.
- The index weights AA-Paperclip (document extraction as one decision) at 30%, with five other evaluations at 10–15% each.
- Accuracy counts the top-probability option as the answer, and the page also reports Brier scores, time per task, and cost per 1M tasks.
@undefinedKi on X
The post says Anthropic published the onboarding material it gives new hires, a framework called the 4D model: Delegation, Description, Discernment and Diligence. Its point is that prompting is only one quarter of working well with AI, and the rest is picking the right work, judging the output, and owning the result.
- Prompting falls under Description, so it is one quarter of the framework; the other three cover choosing work, evaluating output, and taking responsibility for it.
- New hires are trained on common model mistakes so they can review AI work more effectively, and the training continues as "ever-boarding" after onboarding.
- Anthropic made the full material public last week, free, with a certificate, under a Creative Commons license.
Shopify grew its internship program tenfold to recruit developers who’ve grown up with AI and to gauge their coding and judgment in real projects. It evaluates candidates across three modes—no AI, AI optional, AI mandatory—to test hand-coding skills, tool discernment, and AI-driven development.
- Shopify grew its internship program from ~100 to over 1,000 spots, betting on junior talent instead of cutting it in the AI era
- Candidates are tested in three modes—no AI, optional AI, mandatory AI—to assess raw coding skill, judgment, and AI-driven building
- AI has collapsed grunt work like unit tests, refactoring, and learning new frameworks into near-instant tasks, freeing engineers to prototype more boldly
- Even Shopify admits there's no proven formula yet for which skill mix (manual coding vs. AI fluency) actually predicts job success